<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
"http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<meta name="generator" content=
"HTML Tidy for Cygwin (vers 1st September 2003), see www.w3.org">
<meta http-equiv="Content-Type" content=
"text/html; charset=us-ascii">
<title>Makumba design issues</title>
<link rel="stylesheet" type="text/css" media="all" href="main.css">
<script type="text/javascript" src="main.js">
</script>
</head>
<body>
<script type="text/javascript">
   makeNavBar("");
</script>
<h1>Makumba Design Issues</h1>
<p>vers&#227;o $Revision$, November 2000, in Stockholm</p>
<p>This document continuously presents the state of the Makumba
project. It is intended for Makumba designers and implementors. The
idea is to keep all knowledge about Makumba in one place and not to
have it spread in all sorts of files. The spec will be maintained
in parallel. As different from spec, this document also includes
past ideas and future visions.</p>
<p>Recently design issues are kept in <a href=
"http://bugzilla.makumba.org">Makumba's bugzilla</a>, with <a href=
"http://bugzilla.makumba.org/buglist.cgi?product=Makumba&amp;target_milestone=designDiscussion&amp;bug_status=NEW&amp;bug_status=ASSIGNED&amp;bug_status=REOPENED">
designDiscussion milestone</a>.</p>
<p>The only information that is not in this document are the
introdutory text, the <a href="makumba-spec.html">spec</a>,&nbsp;
<a href="api/org/makumba/package-summary.html">the API
documentation</a>, the <a href="tasks.html">tasks</a>, and the
<a href="SQL-drivers.html">supported databases documentation</a>. 
<!--
          (need linked summary at the beginning and
          number-indepedent A NAMEs. automatic section numbering seems impossible in
          Composer) </p>
          --></p>
<h2>1.&nbsp;<a name=
"Principles/philosophy">&nbsp;</a>Principles/philosophy</h2>
<p>(see <a href="#Makumba%20philosophy">Cristi's Philosophy
mail</a>)<br>
pre-packaged BEST features are very different from professional<br>
- no fine-grained control of all details<br>
- a feature is available on the most generic sense with no concern
for speed or space<br>
- few features are available, only the essentials<br>
- makes it easy to learn and develop<br>
- not simple-to-complex learning because an advanced user might use
complex features that a novice won't understand</p>
<p>non-professional<br>
- total control<br>
- feeling of independence<br>
- no general purpose (taught in school) commercial technology
exists for<br>
data-driven web apps (hack over database systems to get to the web
or vice versa)</p>
<p>next (visionary) system will come by design refactoring of the
present one<br>
&nbsp;</p>
<h2>2. Professional Data-driven Web technologies</h2>
<h3>2.1&nbsp;<a name="Lasso">&nbsp;</a>Lasso (<a href=
"http://www.blueworld.com">blueworld.com</a>), its LDML
language</h3>
<p>we didn't manage to run Lasso server on the laptop, but we could
look at language examples<br>
LDML looks very much like what we want for Makumba<br>
observations learned from Lasso:<br>
- one can have more than one presentation language<br>
- any language can have an XML description (whatever the use)<br>
- html editors can be used to edit a good language, and if the tags
are like &lt;..&gt; the html editors will not rewrite them (see
difference between LDML and object LDML<br>
- LDML is very much programming (rather than presentation)
oriented<br>
- too many tags<br>
- nice simple logging mechanism, but it shouldn't be at
presentation layer, or should it?<br>
- lasso seems to be a one-table search thing<br>
- but this is not a problem since you can have views for each join
you can think of</p>
<h3>2.2&nbsp;<a name="CFML">&nbsp;</a>CFML (Cold Fusion Markup
Language, <a href="http://www.allaire.com">allaire.com</a>)</h3>
<p>- SQL dialect dependent<br>
- but more search power<br>
- named queries look interesting<br>
&nbsp;</p>
<h3>2.3&nbsp;<a name="Webmacro">&nbsp;</a>Webmacro (<a href=
"http://www.webmacro.org">webmacro.org</a>)</h3>
<p>Looks like <a href=
"http://best.nada.kth.se:8080/jml/jml.cgi/index.jml">JML</a>/<a href="http://java.sun.com/products/jsp">
JSP</a>. Just a different syntax and probably different compilation
and error messages. But that doesn't make it much better.</p>
<h3>2.4 General observations</h3>
<p>both <a href="#CFML">CFML</a> and <a href="#Lasso">LDML</a> miss
the generation of the default presentation because they don't
describe the data anywhere.</p>
<p>an improvement of CFML and LDML over stuff like <a href=
"http://www.php.net">PHP</a> or JSP is that there is better
integration with the database and less db connection installation
overhead</p>
<p>both have server-side includes</p>
<p><a name="Tango">&nbsp;</a>Tango (<a href=
"http://www.pervasive.com">pervasive.com</a>) is a dead product but
it makes a nice distinction (presentation, data model and business
logic)</p>
<h2>3. Trainee orientation</h2>
<p>Makumba should be oriented in helping trainees to learn it fast,
install it easily and start coding with as few problems as
possible<br>
- removing installation overhead<br>
- tomcat, jdk, database engine, etc., should be pre installed and
packaged<br>
- <a href="http://www.mysql.org">MySQL</a> as database for
windows<br>
- CD or zip with the pre installation and example data (e.g.
complete general tables)<br>
- CVS as overhead during trainee startup (then very useful)<br>
- we don't necessarily want/have the skills from open-source. Not
all trainees need help there, but it's a good diversification of
opportunity.</p>
<p>Igor case: hindrances of the BEST development environment<br>
- a lot of good will<br>
- poor connection<br>
- different locations, slow email communication<br>
- not yet enough experience<br>
- initially a victory (odbc4 driver)<br>
- when the new problem came in (message board), local devel
stopped</p>
<p>Student organisation identification with an ideal<br>
- professional software devel (the spec document, framework that
was promising to comply with changes in spec, etc)<br>
- open source (Makumba), no release yet because of prestige. Not
all of us have to be open source programmers. Some trainees will
only get to do Makumba code development and maintenance, some
others will get to work on spec and internals in the open-source
process.</p>
<p>(kosice) meeting preparation with hands-on examples (low
overhead should help this)</p>
<h2>4. Reflective&nbsp;<a name="history">&nbsp;</a>history</h2>
<p>(this is working information for now, but might be later
"demoted" as an annex)</p>
<h3>4.1&nbsp;<a name="firstVersion">&nbsp;</a>first version (March
1999)</h3>
<p>- abstract, DB and presentation data description<br>
- only had presentation implemented</p>
<h3>4.2 second version (Xalara in&nbsp;<a name=
"Athens1999">&nbsp;</a>Athens July 1999)</h3>
<p>- re-organised directories in package like manner<br>
- some "redundant" features: templates, one-to-many and
many-to-many tables. redundant features complicate the language
while they can actually be implemented with the existing ones<br>
- we were still puzzled about search<br>
- we believed that there is a ideal presentation layer and we tried
to find it<br>
- what we still like: general idea of record description,
replication fields, record title, pointer, set, general type
correspondence, FIXED, NOTNULL<br>
- implemented the first version based on these ideas</p>
<h3>4.3&nbsp;<a name="afterAthens">&nbsp;</a>after Athens</h3>
<p>- proposed searches in separate files. still not clear what to
do with arguments<br>
- still not clear how to select fields from the search<br>
- attempt a second presentation layer; quite cryptic. See Annex
1<br>
- "new database layer" attempted to model all SQL SELECT concepts
in Java objects (Expression and Filter)<br>
- presentation toolkit and other upfront design OOP ideas, for
generating any kind of content, etc...<br>
- ODMG database layer proposed OODB-like principles for the API<br>
- JSP taglib-based presentation layer. See Annex 2<br>
- OQL discussion - resulted in Autumn 2000 implementation of
OQL<br>
&nbsp;we liked the join in the FROM. it "moves" the join from the
WHERE<br>
&nbsp;clause, which is mostly for filtering</p>
<pre>
    SELECT DISTINCT s FROM /best/student s, s.education e,e.studyfield st
    WHERE st.name="Chemestry"

</pre>
<p>joins in the implicit relations?<br>
<tt>/best/minerva/student.minerva m(s)</tt></p>
<h3>4.4 Lost in <a name="KosiceJuly2000"></a>Kosice July 2000</h3>
<p>- attributes are kind of constraints<br>
- not table, but object-oriented<br>
- lists as a clear difference from sets<br>
- inheritance rather than inclusion<br>
- multilanguage suport in text literals<br>
- title is the first non_pointer field<br>
- clean up of syntax<br>
- business logic<br>
- actors<br>
- authentication mechanism<br>
- "extension"<br>
- administration mechanism<br>
- new field constraints UNIQUE, ENCRYPTED<br>
- minimum and maximum char length [a..b]</p>
<h3>4.5 <a name="afterKosice2000"></a>after Kosice (ML
discussions)</h3>
<p>- if we define inline types, we don't have to think of complex
sets and 1:1 pointers anymore. this simplifies the language<br>
- Toto's Business Logic <a href=
"#Toto's%20BL%20sources%20after%20Kosice">sources</a>.<br>
- fieldname: int { default
longhish_shit_that_is_not_comfortable_to_repeat, a, b, c }<br>
- now as default for date<br>
- interfaceLanguage in actor profile<br>
- %localize(lang1, lang2) at presentation level<br>
- dot instead of / in the class name. capital for type name as
style rule. OQL becomes:<br>
<tt>SELECT DISTINCT s FROM best.Student s, s.education e,
e.studyfield st</tt><br>
<tt>WHERE st.name="Chemistry"</tt><br>
- data ecryption not one way<br>
- spec shouldn't talk about SQL, or specify that it is an
example</p>
<h3>4.6 <a name="Coimbra2000"></a>Coimbra October 2000</h3>
<p>- tried to make sense of all stuff and set short-term and
long-term goals<br>
- see design issues below, plus annexes 4 and 5<br>
- we ended in a frustrated note due to too short meeting time. But
overal the meeting was very productive.</p>
<h3>4.7 <a name="AfterCoimbra2000"></a>After Coimbra (Stockholm, ML
discussions, stuff done,...)</h3>
<p>- ODBC and mSQL (temporarily) not supported<br>
- Implemented OQL (with <a href="http://www.antlr.org">antlr</a>,
fixed their OQL grammar, submitted it, got no reply yet)<br>
- bought makumba.org domain name<br>
- made a database copy utility (a good ground for future
replication, transfer between different databases, different SQL
servers, cleaning historical table names)<br>
- moved some Toto's own methods (presumably most frequently used by
a app developer) to a prominent place (metadata.db.Database) for
everybody to use<br>
- webmasters looking for alternatives (PHP, Perl, Apache server
side includes...)</p>
<h3>4.8 <a name="Balatonf&#246;ldvar"></a>Balatonf&#246;ldvar
November 2000</h3>
<p>- subselects in MySQL not fully supported<br>
- realised that database-level indexing improves performance
significantly<br>
- made OQL work in minerva student search<br>
- compiled and packaged a set of programs (17 megs) needed to run
an application (in this case Minerva, working on a non-programers'
laptop) No need to set any PATH or CLASSPATH. Only needed change
might be to enlarge the memory of the command.com but we can
provide proactive assistance for that<br>
- confronted webmasters with jsp --&gt; enthusiasm!<br>
- made BEST realize of Makumba's importance to the organisation /
raised the interest by introducing Karamba!<br>
- markeTeam takes over the PR campaign- promotion of Makumba as an
open source project<br>
- eliminated 'dot convention' - no more dots in table names</p>
<h3>4.9 <a name="AfterBalaton"></a>After Balatonf&#246;ldvar</h3>
<p>- eliminated the need to manually translate names of fields and
tables to avoid conflicts with SQL keywords on some database
engines (perticularly mySql is very sensitive, postgres more
tollerant)<br>
- added a feature to shorten table names (Postgres has limit to 32
characters, MySql no problems)<br>
- documentation clean-up<br>
- updating this doc, API spec</p>
<p>- found a nice model for Rapid Application Development</p>
<h3>4.10 <a name="Rome2000"></a>Rome December 2000</h3>
<p>- First test of a bigger application with the latest version of
Makumba.<br>
- Presentation to webmasters, they love it, we need to make them a
distribution for them to play with it.</p>
<h3>4.11 <a name="AfterRome"></a>After Rome</h3>
<p>- Writing the <a href=
"makumba-spec.html#Makumba%20JSP%20tags">Makumba JSP tag
specification</a>, finalizing the taglibs.<br>
- New Makumba DD language implementation postponed (and have it
done later with antlr)</p>
<h3>4.12 <a name="Weekend_in_Stockholm"></a>Weekend in Stockholm,
December 2000</h3>
<p>The objective was to get to the minimum powerful makumba to be
able to implement Karamba</p>
<p>Solved database config files</p>
<p>Decided to implement types in pointers</p>
<p>Replication when DD changes. What happens to the LBG copies? How
to detect DD and logical changes?</p>
<p>Dillemas about Archive and Document. We didn&#8217;t know how to
organize the DDs, and how to set the access rights.</p>
<p>Dillemas about static (database registered) and dynamic
(application dictated) access rights</p>
<p>Application template, components (maybe configurable by
webmaster), maybe new tags, skins around them</p>
<p>DD-parallel code, express there all we wanted to express in the
DD Business Language for now. Such classes will be called by the
taglib directly (no more JSP parallel code). If a class
doesn&#8217;t exist, a default one is used. If it exists, it is a
sublcass of a powerful toolbox (refactored from needs and
experience) class that knows</p>
<p>- rigts set (static, or by OQL)</p>
<p>- decorated (pointer) field attribute</p>
<p>- unique field attribute</p>
<p>- !unique constraint OQL expression</p>
<p>- computed fields</p>
<p>- unique and notnull to implement directly in SQL</p>
<p>(cristi toilet) 3 tier application: servlet container,
DD-parallel code, database. Will improve development speed</p>
<p>- JSP calls DD-parallel methods of the kind: &#8220;do(type,
operation, data) in the main tables</p>
<p>- DD-parallel code calls methods of the same kind in the tables
and subtables, after security checks, etc.</p>
<p>We realised that insertUnique just comes from the unimplemented
unique constraint</p>
<p>We have a clear idea of what an application is. Application come
in bean-like components (maybe will be implemented as java
beans?)</p>
<p>- DDs, with a field that indicate which instance of the app do
we talk about (name)</p>
<p>- DD-parallel files, written in general</p>
<p>- JSPs, maybe with an argument</p>
<p>- Maybe a special tag, say &lt;karamba:todolist
specificArguments../&gt;</p>
<h3>4.13 After Stockholm</h3>
<p>package name metadata &#8211;&gt; org.makumba</p>
<h2>5. <a name="DesignIssues"></a>Design issues</h2>
<p>Issues are divided in current problems (5.1), short term
objectives (5.2) and long term guidelines (5.3). Every such
timeframe is divided in General principles, Data description, BEST
(organisational/Business) Logic (BL), Java API, Presentation and
Development support. Development support describes objectives for
fast development and change by experts, trainees, webmasters and
normal users.</p>
<p>The new spec will be written by Stefan and comply to 5.1 and
5.2<br>
The long term spec will be developped based on 5.3</p>
<h3>5.1 Current problems</h3>
<h4>5.1.0 administrative stuff</h4>
<p>as a testbed for new taglibs and API we can use few pages from
minerva, see how it feels, what webmasters feel about it.</p>
<p>ANT distribution</p>
<h4>5.1.1 general principles</h4>
<p>get the business logic in DD-parallel code</p>
<p>finish makumba!!!</p>
<h4>5.1.2 Data description</h4>
<p>unique, unique constraints, list, fixed, default
values&#8230;</p>
<h4>5.1.3 BL (BEST/Business Logic)</h4>
<p>specify and implement DD-parallel code</p>
<p>- make use of strong context provided in the data description
files and in the presentation. Making use of context leads to less
initialization code, less installation overhead<br>
- API growth by pattern discovery. When typical features are
identified, they are included in the toolbox</p>
<p>- decoration will come by refactoring</p>
<h5>Actors, administration security models</h5>
<p>- authentication: !auth=field_name. the class of the field has
to have at least one authenticate* method</p>
<p>- shouldn't the authentication field be always UNIQUE, FIXED,
NOTNULL?</p>
<p>- shouldn't !admin be a method, like !title and maybe other
commands... ?</p>
<p>- an actor has an authentication mechanism or is a decoration of
it</p>
<p>- authenticating the extension just needs to authenticate the
base<br>
!base=class_name</p>
<p>- how to express in the language that 2 objects of different
classes have the same auth?</p>
<h4>5.1.4 Java API</h4>
<p>implement the Database config files with names like</p>
<p>localhost_mysql_makumba</p>
<p><span lang="SV">malaka.best.eu.org_pgsql_minerva2</span></p>
<p>table alteration for the whole DB</p>
<p>There is no way of setting a field to null. There should be a
Null value defined at the abstract level.</p>
<p>implement database catalog and full database copying</p>
<p>subtables (ptrOne, set*) should accept data in insert and return
data in read. Data should be part of the Dictionary for ptrOne and
Enumeration for sets. This looks like it can be written at the
metadata.db.level</p>
<p>delete with deletion of internal sets and pointers...</p>
<p>list, set, bag API</p>
<p>profiling of the code once everything is implemented</p>
<p>change /best/student -&gt; best.Student, relative:type from the
same package in reference and include</p>
<p>transaction</p>
<p>text type</p>
<p>notification services</p>
<p>DBSV autogeneration</p>
<p>keep configuration in a special table</p>
<p>database copying starting from a type</p>
<p>implement title as first non-pointer field</p>
<p>implement full-text search</p>
<p>parse default values</p>
<p>implement LIST, FIXED, NOTNULL, UNIQUE</p>
<p>Database bugs:</p>
<ul>
<li>some db engines return Float at getObject when they read
ints... Should be a cast to Number (rather then Integer), then a
intValue()</li>
<li>msql.Database does not create a connection pool, which will
make concurrent accesses fail</li>
</ul>
<p>Other db problems<br>
- find other Windows development DB engine (recompile Postgres with
cygwin32 ?)<br>
<![if !supportLineBreakNewLine]><br>
<![endif]></p>
<p>metadata.abstr.printer.subtablePrinter prints the description
after all the fields of the subtable. should be before</p>
<h4>5.1.5 Presentation</h4>
<p>finish taglib</p>
<h4>5.1.6 support for development</h4>
<p>implement RAD interface development</p>
<p>existing: support for pre-Makumba existing tables: changes of
table and field names. nice stuff</p>
<h3>5.2 Short term</h3>
<h4>5.2.1 General principles</h4>
<p>- the present Makumba code can be kept (taglibs, tables and
searches can coexist or be based on it)<br>
- maybe logging</p>
<h4>5.2.2 Data description</h4>
<h5>extension, include?</h5>
<p>- non-pointer inclusion of types</p>
<h5>DD language</h5>
<p>- a well-formatted metadata file should somehow mention its name
for easy citation</p>
<p>- classes rather than tables. objects rather than table records.
this needs discussions... instances of applications, etc. See OODB
ideas (Annex 6)</p>
<p>- Constraints, field constraints, sets are a constrained bag</p>
<p>- field seperator in DD files (newline too strict?)</p>
<p>- default values</p>
<p>- title as expression rather than field</p>
<h5>Decoration</h5>
<p>- if you mention an decorator object (say minerva.student) and
the base object exists, the mentioned object is automatically
created.</p>
<p>- if the base object doesn't exist, it is created first.</p>
<p>- editing/deleting the base object can be a default right?</p>
<p>- should based types be returned together with the base ?</p>
<p>- should the base object be somehow notified when a wrapper is
created?</p>
<p>- Toto: Can it be recursive (i.e. decoration of decoration) ?
With the current definition, I don't think so. Moreover we don't
really need it, and it may be more difficult to handle...</p>
<p>- Toto: Can an extension be authenticated in a different way
than its base ? (i.e. !authenticate)</p>
<p>- Toto: (see <a href=
"#Toto's%20BL%20sources%20after%20Kosice">annex 3</a>) Does
/best/minerva/student inherit the !admin from /best/student
implicitely (i.e. Helpdesk) ? I would say yes. And in case !admin
is defined in /best/minerva/student ? Should it be the union of
admins of /best/student and /best/minerva/student or only admins of
/best/minerva/student ?</p>
<h4>5.2.3 BL (DD parallel)</h4>
<h5>Virtual sets</h5>
<p>- virtual sets with arguments</p>
<p>- virtual set with arguments == search?</p>
<p>- virtual set called from another virtual set</p>
<p>- subsets of (virtual) sets, lists of (virtual) sets, etc.
introduce constraints beautifully</p>
<p>- virtual set inside the virtual set (see johnny student)</p>
<p>- static virtual set</p>
<h4>5.2.4 Java API</h4>
<h5>API</h5>
<p>- wrapper managers are a more general concept, should move up
from database. Should be metadata.abstr.WrapperManager or so</p>
<p>Abstract level bugs<br>
- if there is a redirection to a type which is redefined on a lower
family, the lower family type should be obtained. Instead, the same
family type is obtained now</p>
<p>Save OQL searches for future use (with different parameters).
Maybe keep them as stored procedures</p>
<p>Database bugs</p>
<ul>
<li>at boot time, the ptrIndexJava computes the max index. This
means that the index values of deleted records might be reused,
which is bad for replication</li>
<li>- unique id generation should use more the power of the SQL
engine (now it's java level). CREATE SEQUENCE seems to be the
standard way. This would also allow remembering the index value
from one DB boot to another</li>
<li>generation of creation date and last modification date should
use TIMESTAMP facilities of the SQL engine</li>
<li>before altering, tables should be backed up</li>
<li>there should be some factory method that produces a pointer for
each type of database</li>
</ul>
<p>OQL</p>
<p>- if just a label is mentioned in where/select then the primary
key field of that table is used</p>
<p>- parameters $number</p>
<p>- NPE bug if no label and no type exists, a "best" table is
mistakenly considered</p>
<p>- best.Student is considered, should be error... or is this
windows?</p>
<p>- unmentioned labels should not generate joins</p>
<p>- multiple objects &amp;expressions in SELECT</p>
<p>- expressions in select and where are equivalent, so it's easy
to have joins generated there</p>
<p>- subselects can be implemented, inheriting the labels of their
containing selects. how about the SQLs that don't support
subselects?</p>
<p style='margin:0cm;margin-bottom:.0001pt'>- SQL verbosity
level</p>
<h4>5.2.5 presentation</h4>
<p>- make sure that webmaster can't change the access rights of a
page<br>
- support modification in html editors?</p>
<h4>5.2.6 support for development</h4>
<p>- not much added for programmers (maybe the new generated
classes best.Student().insert(data))</p>
<h3>5.3 Long term</h3>
<p>Design will continue from here.</p>
<h4>5.3.1 general principles</h4>
<p>- data description modification: try to detect and signal all
presentation files and java files that might be affected<br>
- java code in the data-description file, or modification of
generated java file out of the description?<br>
- problems with modification of original data description when
default code generation is used at presentation and data level (and
then we let the programmer/webmaster alter the code). Toto proposes
class inheritance for the generated Java files (you extend the
generated file, which stays untouched)<br>
- dealing with data description changes: there will be extensive
change detection (i.e. analysis that takes into account the
previous version) and automatic fix of database tables, database
code and presentation code (this supports automatic preservation of
existing data and code)<br>
- types of data structure change: field name change, field type
change, field move in another table, ptr -&gt; set transformation
(and back?)<br>
- transactions, table and record locking<br>
- logging. what level? actor? db?<br>
- XML dump and import. DTD<br>
- data travels in XML rather than Hashtables<br>
- allow the server to receive data described by a certain DD via
email<br>
- support for backup and versioning at db level<br>
- multilingual support tables, enumerated type string, default
values (mb also presentation layer)<br>
- attachement conversion... what level? presentation? preprocessing
and save as text or html??<br>
- replication... keep date of last replication. transitivity. how
does the data structure change support affect replication? quite
much... replication of data descriptions between sites<br>
- selective replication: a location could get replicated locally
data that is more related to its people<br>
- support again more DBs (ODBC, msql, etc)<br>
- looks like we can't count on ODBC (LIKE, close-on-exit, different
versions...). on the other hand, it's cool to be able to generate
excel views of the data. we could support it partially.<br>
- remote database??? Old description: to support remote access to a
database, we need:</p>
<ul>
<li>some remote wrappers: metadata.remotedb.Database,
metadata.remotedb.Table, etc. Such wrappers will be remote objects
and their implementation will keep a reference to the normal
objects (metadata.db.Database, metadata.db.Table) and call their
methods when needed</li>
<li>at the abstract level, the data has to be serializable</li>
<li>some sites might not have the metadata files. The remote tables
should be able to pass their RecordInfo (which is
serializable)</li>
<li>the remote and local Db and especially tables should implment
some common interface so they can be used interchangeably</li>
</ul>
<p>- need notification services for insert and update. different
actors may choose to be notified when a new record is inserted or a
certain record is changed. only some such records may be relevant
for the respective actor, so filters may need be installed. access
rights problems exist here (i.e. I may be notified only of records
I am allowed to see)<br>
- should pointers be represented as 1 or as 2 SQL longs (java
ints)<br>
- save search parameters for future refinement and use&#8230;<br>
- relations with other buzz technologies: Java Beans, WAP</p>
<h4>5.3.2 Data description</h4>
<p>- change parser to understand ENCRYPTED. field encryption not
one way<br>
- surely implement on-the-spot types (to replace ptrOne and
setComplex)<br>
- omit ptr keyword<br>
- minimum and maximum char length [a..b]<br>
- fieldname: int { default
longhish_shit_that_is_not_comfortable_to_repeat, a, b, c }<br>
- now as default for date<br>
- database-platform independence (compilation up to stored
procedures for higher performance)</p>
<h4>5.3.3 BL (BEST/Business Logic)</h4>
<p>- actors, rights and other constraints, erros if constraints are
not respected, actions (field sets, etc)<br>
who : action (fields) precondition, operations, postcondition</p>
<p>- actions (understood as method adjustments) are actually
expressions of DB redundancies? Toto: constraints can also be
expressed in actions</p>
<p>- the meaning of set in argument lists and method results seems
to be different from the meaning of set in sets and virtual
sets</p>
<p>- implicit assigments might be difficult to "read"</p>
<p>- methods with no actor cannot be called from presentation</p>
<p>- static methods</p>
<p>- look again into best/johnny/student (see annex 2)</p>
<p>- looks like constraints don't have to sit in the method. but
then, how do you ensure that a consistent state is kept?</p>
<p>- we could say "these 2 op form an atom. don't check consistency
between"</p>
<p>- how to name decorator base when used in argument lists?</p>
<h4>5.3.4 Java API</h4>
<p>- should loose importance but should still exist<br>
- ODMG, OODB ideas</p>
<p>- optimize current drivers</p>
<p>- consider also introducing relationships on the database
level<br>
- Stored procedures from OQL or CREATE VIEW ?<br>
- evaluation of the select() should be lazy. That is, the SELECT
should be executed only when the hasMoreElements() or nextElement()
methods of the returned enum are called<br>
- there could be a form of Table.insert() that gets a record set
(not just a record) as argument. At SQL level this could generate
an INSERT INTO table SELECT ... if the recordset comes from a
search _in_the_same_database_</p>
<p style='tab-stops:100.8pt'>DBSV auto generation?</p>
<p>support for existing tables: besides changes of table and field
names, import structure from SQL tables, changes of type, automatic
change of #alter file, facilities of changing types (such as a
abstract-level int{..=1 ...=0} that might actually be a BIT in the
existing pre-Makumba DB)</p>
<p>it's not clear what will happen with DBSV for an existing
database. A possible solution is to consider it 0 for start
(replication is not on anyway). Before the first replication, a
pointer transformer can be executed on each table, to adjust all
pointer fields</p>
<p>- redirection property files could be eliminated. each record
handler should have an associated redirector class, that has
methods for each redirected type, and a getFamillyName method. Due
to the getFieldMethods(), each handler family has to have a
FieldHandler (i.e. cannot inherit it). That's wrong (e.g.
metadata.sql.odbc and metadata.sql.msql don't need FieldManagers,
they are just empty extensions of metadata.sql.FieldManager)</p>
<h4>5.3.5 presentation</h4>
<p>- XSL is not webmaster-friendly, but XML+XSL= presentation.
Maybe XSL should be generated out of our own presentation
language<br>
- we don't want access rights (field level and actor) in the
presentation<br>
- database-to-presentation tools that provide default views can be
found in:<br>
- IDEs<br>
- database management applications (Access, Oracle 8i)<br>
we should look at those, just the same as we looked at LDML and
CFML<br>
BUT none of these are easily accessible to the BEST webmaster and
BEST programmer on the BEST platform and server in the distributed
BEST environment. any of them can change and the system should
survive<br>
- need to check enhydra.org
http://www.enhydra.org/software/enhydra/index.html<br>
- support modification in html editors<br>
- interfaceLanguage in actor profile. how should it work?<br>
- how should the content in the web pages be localized?<br>
- %localize(lang1, lang2) at pres level<br>
- think of other kinds of presentations: AWT, RTF, XML, WAP,
etc</p>
<h4>5.3.6 support for development</h4>
<p>- the moment you have the profile of an action, you can make a
default JSP page that uses it. that page already has access rights
on it<br>
- installation: one can automatically install ODBC data sources
with odbcconf.exe<br>
- (html and view source) is an epitome of low installation and
initialisation overhead<br>
- provide for trial and error experimentation<br>
- maybe we need to look a bit of how webmasters work (see the
alttabing stuff in the cfml tutorial)<br>
- pay attention to errors, where should they come up and how.
should not be hidden in the source<br>
- manual, reference manual, functional sample applications (FAQ,
support forum, discussion, to-do, calendar, system db, bug db, see
mail about small app ideas)<br>
<![if !supportLineBreakNewLine]><br>
<![endif]></p>
<p>- wizard (sounds too professional)<br>
1 new metadata file<br>
2 links to generated presentation and business logic file<br>
3 the generated files contain comments with explanations,
suggestions and links to tutorial. the form and comments of
generated code can be adjusted by setting a "level of
expertise"<br>
4 the programmer can edit the default presentation and business
logic<br>
- in the general case, altering should be done by ALTER TABLE,
though a convenience java alter should be provided</p>
<h3>5.4 Administrative to-do</h3>
<p>- GPL needs be checked in connection with the antlr OQL grammmar
we use</p>
<h2>Acknowledgements</h2>
<p>Thanks to BEST for trusting us to come up with something better
than <a href="http://notes.net">Lotus Notes</a>.</p>
<p>Thanks to LBGs in Athens (summer 1999), Chania (autumn 1999),
Kosice (summer 2000) and Coimbra (autumn 2000) for their kind
hosting support.</p>
<p>Thanks to Yngve for supporting our web-programming ideas (since
JML in 1996) which were not on the research agenda and even
employing supplementary personnel for their development.</p>
<h2>Annexes</h2>
<p>Annexes present code snippets that might still be useful for
future development</p>
<h3>Annex: <a name="Makumba_philosophy"></a>Makumba philosophy
mail</h3>
<p>Date: Thu, 31 Aug 2000 19:05:26 +0200<br>
From: Cristian Bogdan &lt;cristi AT nada.kth.se&gt;</p>
<p>1. common technology:<br>
- our data is fragmented across more technologies. In Notes, data
is fragmented across more databases, because we don't master the
technology or simply because Notes sucks.<br>
- we need total control over the technology or a very good customer
service from whoever provides it<br>
- in order to achieve a low learning (and maintenance, if we do it)
treshold, it has to be simple, i.e. have as few features as
possible, the minimum necessary for us<br>
- it should be portable to any platform our low-budget needs will
get us to</p>
<p>2. what is the nature of our applications?<br>
- we have web database applications.<br>
- thus the technology should have ready-made building blocks
oriented to fast generation of such applications<br>
- it should also provide the power for webmasters to adjust the
pages to the lowest detail<br>
- same goes for programmers</p>
<p>3. how do our applications evolve?<br>
- we rarely do ready-made applications. normally we have the
minimum necessary to gather some data. this ensures early
availability of a working application, without much efforts from or
pressure on the very-specialised-people (this corresponds to data
description files)<br>
- then (when programmers have time or pressure) we add more and
more logic, or provide ways to enter more data. (this corresponds
to business logic level)<br>
- in the meantime, webmasters (when they have time or when they are
pressed) add more and more layout (this corresponds to presentation
level)</p>
<p>4. problems with commercial technologies<br>
- they usually contain loads of functions to make for the needs of
all big customers<br>
- ... and to obtain lots of speed for very specialised tasks<br>
- if the technology is commercial we should be able to obtain not
only the technology itself but also its customer support and make
sure we can provide for its maintenance overhead<br>
- commercial would usually tie us to a vendor. BEST breaking
relations with the vendor results in problems. ITC starting to hate
the vendor gives other problems<br>
- commercial might tie to a platform or a set of platforms and
porting to a new platform is independent on us</p>
<p>5. "external" technologies (open source or commercial):<br>
- they are used on more than one place so they can be more
reliable<br>
- there is a dedicated maintenance crew<br>
- if we can't find any external, we can get to both these
advantages by<br>
i) making a good design<br>
ii) implementing it<br>
iii) open-sourcing and publishing it in good places (i.e. places we
looked for one and didn't find :)</p>
<h3>Annex: Presentation language as of autumn 1999</h3>
<pre>
&lt;h1&gt;All students:&lt;/h1&gt;
$/best/student
[
    Make a &lt;a href=$newLink&gt;new&lt;/a&gt; student or check this list:&lt;p&gt;
     {
            this is a student:&lt;br&gt;
            name is $(name)[$value] &lt;b&gt;$(surname)[$value]&lt;/b&gt;&lt;br&gt;
            &lt;font size=-2&gt;
            &lt;a href=$editLink&gt;edit&lt;/a&gt;
            &lt;a href=$deleteLink&gt;delete&lt;/a&gt;
            &lt;/font&gt;&lt;br&gt;
            $(birthdate) [&lt;font size=-2&gt;$value&lt;/font&gt;&lt;p&gt;]
            $$[&lt;b&gt;$description&lt;/b&gt;:$value &lt;br&gt;]
    }(&lt;br&gt;)
]
</pre>
<h3>Annex: <a name="taglib-proposal"></a>First proposal of JSP
taglib presentation. Will go to API Spec</h3>
<pre>
&lt;% // give a name to the data to be formatted (there might be better ways to do this...) %&gt;
&lt;%pageContext.setAttribute("std1", someDictionaryFromTheDatabase);%&gt;

&lt;%// I want to format a student for editing, this should open a &lt;form&gt; %&gt;
&lt;mtd:record
   structure="/best/student"
   data="std1"
   submitMethod="editMyself"
   afterSubmit="index.jsp"
   success="&lt;font color=green&gt;Changes done!&lt;/font&gt;"
   failure="&lt;font color=red&gt;Changes not done:  $error&lt;/font&gt;"
&gt;
     &lt;%// I want to format some fields, separated by &lt;br&gt; %&gt;
     &lt;mtd:fields names="name,surname,birthdate" separator="&lt;br&gt;"&gt;
          &lt;%// field description in bold, then :, then value in a form input%&gt;
          &lt;b&gt;&lt;mtd:fieldDescription/&gt;&lt;/b&gt;:
             &lt;mtd:fieldEditValue/&gt;
     &lt;/mtd:fields&gt;

     &lt;%// now I format other fields, a bit shorter but with the same result, the fields are displayed for reading ($value), not as html form fields ($editValue) %&gt;

    &lt;mtd:fields names="military, etc" pattern="&lt;b&gt;$descr&lt;/b&gt;:$value" separator="&lt;br&gt;" /&gt;

    &lt;%//format the addresses for reading %&gt;
    &lt;h2&gt;Addresses&lt;/h2&gt;
    &lt;dl&gt;
    &lt;mtd:set name="languages" separator=""&gt;

       &lt;%// the address kind as a &lt;dt&gt; %&gt;
       &lt;dt&gt;&lt;b&gt;&lt;mtd:fields name="kind" pattern="$value" /&gt;&lt;/b&gt;

       &lt;%// all unmentioned fields, i.e. all but "kind" , as &lt;dd&gt;%&gt;
       &lt;dd&gt;
       &lt;mtd:fields names="$rest" pattern="$name:$value" separator="&lt;br&gt;"/&gt;

    &lt;/mtd:set&gt;
    &lt;/dl&gt;

&lt;%// ok, that's it for this student, this should close the &lt;form&gt; %&gt;
&lt;/mtd:record&gt;
</pre>
<h3>Annex: <a name="Toto's_BL_sources_after_Kosice"></a>Toto's BL
sources after Kosice</h3>
<pre>
-----------/best/auth------------------
username : UNIQUE FIXED char[1..40] ; EN=username
password : ENCRYPTED char[4..40] ; EN=password
code1 : int ; first 5 digits
code2 : int ; last 5 digits

&lt;authenticate&gt;
&lt;arguments&gt;
username
password
&lt;/arguments&gt;
&lt;result&gt;
result : /best/auth (
result .username==username
result .password==password
)
&lt;/result&gt;
&lt;/authenticate&gt;

&lt;authenticateByCode&gt;
&lt;arguments&gt;
code1
code2
&lt;/arguments&gt;
&lt;result&gt;
result : /best/auth (
result .code1==code1
result .code2==code2
)
&lt;/result&gt;
&lt;/authenticate&gt;

&lt;new&gt;
&lt;arguments&gt;
username
password
&lt;/arguments&gt;
&lt;body&gt;
&lt;/body&gt;
&lt;/new&gt;

&lt;edit&gt;
&lt;actor&gt;
this
&lt;/actor&gt;
&lt;arguments&gt;
password
password2 : char[4..40] ; confirm password
&lt;/arguments&gt;
&lt;body&gt;
password==password2 ; wrong password
&lt;/body&gt;
&lt;/edit&gt;

---------/best/student----------
!authenticate=auth
!title=auth.username

auth : UNIQUE FIXED NOTNULL /best/auth

firstname : char[1..40] ; EN=firstname
lastname : char[1..40] ; EN=lastname

address : set
{
kind : UNIQUE char EN={"home addres","study-time address"}
street : char[80]
city : char[80]
counbtry : /general/country
}

&lt;new&gt;
&lt;actor&gt;
anonymous
&lt;/actor&gt;
&lt;arguments&gt;
firstname
lastname
password
password2 : char[4..40] ; confirm password
&lt;/arguments&gt;
&lt;body&gt;
password==password2 ; wrong password
auth.new(firstname+" "+lastname,password)
&lt;/body&gt;
&lt;/new&gt;
-------------/best/minerva/student-------------
!base=/best/student
!admin=set /best/minerva/admin

restriction=set /best/minerva/company ; companies not allowed to see my data

&lt;edit&gt;
&lt;actor&gt;
this
&lt;/actor&gt;
&lt;arguments&gt;
restriction
base
&lt;/arguments&gt;
&lt;body&gt;
&lt;/body&gt;
&lt;/edit&gt;

&lt;view&gt;
&lt;actor&gt;
set /best/minerva/company
&lt;/actor&gt;
&lt;arguments&gt;
base
&lt;/arguments&gt;
&lt;body&gt;
!restriction.contains(actor)
&lt;/body&gt;
&lt;/view&gt;

-------------/best/minerva/admin-------------
!authenticate=auth

auth : UNIQUE FIXED NOTNULL /best/auth


-------------/best/minerva/company-----------
!authenticate=set /best/minerva/company.representative (authenticate.company==this)

name : UNIQUE FIXED char[1..40]

&lt;search&gt;
&lt;actor&gt;
this
&lt;/actor&gt;
&lt;arguments&gt;
languages : set /general/language
interests : set /general/interest
universities : set /general/university
&lt;/arguments&gt;
&lt;body&gt;
actor.paid() ; You are not allowed to search for student : pay the entrance fee first...
&lt;/body&gt;
&lt;result&gt;
result : set /best/minerva/student (
!(restriction.contains(this)
result.student.interest.intersection(interests).size&gt;0
result.student.studies.university.inter(universities).size&gt;0
)
&lt;/result&gt;
&lt;/search&gt;

----------------/best/minerva/company.representative-------------
!authenticate=auth
!admin=set /best/minerva/admin

auth : UNIQUE FIXED NOTNULL /best/auth
company : FIXED NOTNULL /best/minerva/company

----------------/best/johnny/admin-------------------------------
!authenticate=auth

auth : UNIQUE FIXED NOTNULL /best/auth

---------------/best/johnny/student-----------------------------
!admin=/best/johnny/admin
!base=/best/student

season : set /best/johnny/season (
  applic : set /best/johnny/application (
  applic.student==this AND applic.season==season
  )
  applic.size()&gt;0
)

applications(season : /best/johnny/season) : set /best/student/application (
  applic.student==this AND applic.season==season
)

-------------------/best/johnny/season----------------------
activities : set /best/johnny/activity (activities.season==this)
announcementDL : date
applicationDL : date
rankingDL : date

optimise()

----------------/best/johnny/activity----------------------
applications : set /best/johnny/application (applications.activity==this)
ranking : list /best/johnny/application
rankingPlaces : int ; number of accepted people
season : FIXED NOTNULL /best/johnny/season

----------------/best/johnny/application------------------
activity : FIXED NOTNULL /best/johnny/activity
student : FIXED NOTNULL /best/johnny/student
extraAccepted : date

season : /best/johnny/season (season==activity.season)

&lt;new&gt;
&lt;actor&gt;
set /best/johnny/student
&lt;/actor&gt;
&lt;arguments&gt;
activity
&lt;/arguments&gt;
&lt;body&gt;
season.applicationDL.after.now ; Too early to apply
season.rankingDL.before.now ; Too late to apply
student=actor
student.applications.append(this)
&lt;/body&gt;
&lt;/new&gt;

---------------------minerva.student.register.jsp------------
&lt;metadata:new type="/best/student" response="registerResponse.jsp"&gt;
&lt;! register creates the student ; it has a response form...&gt;

--------------------minerva.student.login.jsp-----------------
&lt;metadata:login type="best/minerva/student"&gt;

&lt;! we could have different kinds of login :
&lt;metadata:login type="/best/johnny/student" type="http" method="authenticateByCode"&gt;

--------------------minerva.student.index.jsp----------------
&lt;metadata:actor type="/best/minerva/student" login="login.jsp"&gt;
&lt;! by default, login="login.jsp"&gt;
&lt;! if the /best/student exists but not /best/minerva/student, there should be a "base-extension" notification (specified in login.jsp)&gt;

&lt;metadata:view&gt;

------------------minerva.company.student.jsp--------------
&lt;metadata:actor type="/best/minerva/company"&gt;

&lt;metadata:retrieveObject type="/best/minerva/student" name="student"&gt;
&lt;! retrieve the student from the http arguments&gt;

&lt;metadata:view object="student"&gt;
</pre>
<h3>Annex 4. Thoughts about the generated stuff and its dynamix
(Coimbra)</h3>
<p>DataDescription generates and compiles JavaFile<br>
ModifiedJavaFile extends JavaFile<br>
Presentation calls ModifiedJavaFile</p>
<p>DataDescription generates Presentation</p>
<p>GeneratedPresentation is insert, edit, delete, view-all
pages</p>
<p>GeneratedPresentation is generic -&gt; the only reference to
DataDescription is the name</p>
<p>OR: GeneratedPresentation is not generic, it refers
DataDescription fields, especially title, pointer and set fields.
it is fully commented</p>
<p>Problems:<br>
1. what if somebody wants to use another structure than IEDVA? what
if somebody wants different filenames?</p>
<p>URL is part of the presentation so it can be changed
creative<br>
two different pages might do essentially the same IEDVA
operation</p>
<p>immediate answers<br>
- tutorial and scrap areas<br>
- default area (if stuff is to be changed it is copied in other
areas)</p>
<p>2. What happens to the presentation when DD changes?<br>
professional solution: semantic analysis of Makumba tags in
Presentation</p>
<p>low profile solution: identify the pages that refer to the DD
and list them</p>
<h3>Annex 5. Thoughts about how to present suggestions to novices
in generated code</h3>
<p><tt>/* name is the title fields, so we format it differently
*/</tt><br>
<tt>&lt;b&gt; &lt;mtd:field name="name"&gt;&lt;/b&gt;</tt></p>
<p><tt>/* we put the title first, it is the field "name", the first
non-pointer field in the data description, be aware that if you
change the DD, you might change the tilte field*/</tt><br>
<tt>&lt;b&gt; &lt;mtd:title&gt; &lt;/b&gt;</tt></p>
<p>novice-expert modes for comment level</p>
<p><tt>/* we could have done all this by saying &lt;mtd:record&gt;.
this would use all the above defaults */</tt></p>
<h3>Annex 6. <a name="OODB_proposal"></a>OODB proposal (cristi, Feb
2000)</h3>
<p>needs review</p>
<h4>Object DB update</h4>
<p>Unlike classical relational DB operations (based on insertion,
update and deletion into tables), object repositories are based on
writing down in the database all changes that occur to the fields
of the objects involved in a transaction. Instead of thinking in
terms of tables, the programmer can then only consider the objects
involved, and the transactions necessary for their state to be
consistent.</p>
<p>Metadata already has concepts from object databases, including
pointers, sets, etc, and already defines a database scheme, which
makes it easy for it to support a more object-like update
mechanism.</p>
<p>Metadata can actually be brought closer to Object database
management and <b><a href="http://www.odmg.org">ODMG
standards</a></b> by:</p>
<ul>
<li>supporting inheritance (easy, not urgent)</li>
<li>supporting method declaration (and mapping it to ODL)
(difficult, not urgent)</li>
<li>specifying how the metadata definition maps to ODL (object
definition language) (not urgent)</li>
<li>using ODMG's <b>OQL</b> or a mapping of it to specify queries
(quite urgent)</li>
</ul>
<p>If APIs look like below, metadata/db can be implemented using
any ODMG (or later JDO) implementation (e.g. Sun's <b><a href=
"http://java.sun.com/beans/">Java Blend</a></b> already
implementsODMG, we could try to see if we can't implement
metadata/db with it already, there is a 30 day trial).</p>
<p>To get faster access to fields, classes can be generated
automatically out of the metadata files. In order not to need JVM
changes or class file postprocession, field changes/reads can be
accomodated with methods that follow the <b><a href=
"http://java.sun.com/beans/">Java Beans</a></b> property name
patterns: setXXX(Object) and Object getXXX(), where XXX is the
field name. This patten can co-exist with the present
object-Dictionary approach.</p>
<h4>Proposal</h4>
<p>All records are represented in memory as DBObjects (can extend
Dictionary). To pass a pointer to an object, one just has to pass
that DBObject. Such DBObjects can be registered into the Database
(which will lead to an INSERT), and some of them can be given
special names. Queries made in only one table return collections of
DBObjects, and pointer and set fields will come as DBObjects.</p>
<p>DBObject methods for reading :</p>
<ul>
<li>Object Object get(field)</li>
<li>Dictionary get() returns a clone, without the special
fields</li>
<li>Timestamp getLastModificationDate()</li>
<li>Timestamp getCreationDate()</li>
</ul>
<p>Fields are returned as:</p>
<ul>
<li>their normal data for int, char, text, date</li>
<li>other DBObjects for pointers</li>
<li>DBSets (Collection?) for sets, with specific operations for the
respective type of set</li>
</ul>
<p>Modifications (incl insertion) and destructions are performed
inside transactions. Every thread is part of at most one
transaction. A thread can join an already existing transaction. If
a write operation is invoked, but no transaction is associated with
the thread, a transaction consisting of only that operation will be
performed.</p>
<p>Write operations will cause one INSERT or UPDATE for each
affected object inside a transaction. INSERT is performed on an
object with no pointer (and leads to setting that pointer), and
UPDATE otherwise.</p>
<ul>
<li>put(field, value)</li>
<li>put(Dictionary) updates only the mentioned fields</li>
</ul>
<p>Object destruction: DBObject has a delete() method. All ptrOnes,
setInt, setChar, setComplex will be deleted (in the same
transaction).</p>
<p>A transaction can lock certain DBObjects for reading or writing.
A transaction can be cancelled (abort, nothing is written to the
database), or declared finished (commit, all changes are written).
Also, the transaction can finish and re-start immediately, without
releasing any locks.</p>
<h4>Implementation considerations</h4>
<p>At JDBC level, the Database will employ one Connection per
transaction. Connections can of course be recycled.</p>
<p>The database has a cache, having the composite key [RecordInfo,
ptr]. The cache contains data from the respective table (eventually
in an array Object[] for each record).</p>
<p>The DBObject contains just a reference to the cache entry where
its data stays. When the DBObject goes out of scope, the cache is
notified that the respective entry has one less usage. A cache
cleaner passes regularly and deletes cache entries which are not
used.</p>
<p>To achieve this cache cleaning, based on the DBObjects going out
of scope, one has to build a DBObject instance whenever a query
result is returned, or pointer de-reference is made. That means
that two DBObjects can represent the same thing, so changing a
DBObject might affect another DBObject. To test if two DBObjects
represent the same record in the database we can have a method such
as</p>
<ul>
<li>boolean isIndenticalWith(DBObject)</li>
</ul>
<p>To avoid this, one has to use <b><a href=
"http://java.sun.com/java2/">Java 2</a></b> (JDK 1.2) weak
references. Using java 2 will also bring more standardization with
the Collection classes. But for that, we need to upgrade to <b>Java
Web Server 2.0</b> (which might solve other problems we have now,
such as page recompilation), currently Johnny and Minerva use Java
Web Server 1.1.</p>
<p>Locking can be implemented at Java level or at database
level.</p>
<p>For many of the classes (Database, Transaction) we could
directly implement the ODMG interfaces. This will create a
ODMG-like "metadata binding", which will give a standard value to
metadata.</p>
<p><b>OODB</b> <b>discussion</b> (Cristi and Stefan in Technico).
This discussion probably came from the data-code relationship... In
normal DBs, all objects of a type are accessible by a simple
select. In OOP, objects are accessed thru references. This looks
strange. Proposal to implement data inheritance with 2 extra
columns (one table name and one pointer) in each table that points
to a record containing the extension columns in separate extension
tables. For example, a committee might request an extension to
their archive, which in principle will behave like all other
archives. This leads to thoughts about application inheritance,
etc. In an OODB approach, viewing a table actually would view one
single virtual set.</p>
<h3>Annex 7. <a name="JDBC_type_coding"></a>JDBC type coding</h3>
<p>useful for development</p>
<table border="1" cellspacing="3" cellpadding="0" cols="2">
<tr>
<td>
<p>BIGINT<br>
BINARY<br>
BIT<br>
CHAR<br>
DATE<br>
DECIMAL<br>
DOUBLE<br>
FLOAT<br>
INTEGER<br>
LONGVARBINARY<br>
LONGVARCHAR<br>
NULL<br>
NUMERIC<br>
OTHER<br>
REAL<br>
SMALLINT<br>
TIME<br>
TIMESTAMP<br>
TINYINT<br>
VARBINARY<br>
VARCHAR</p>
</td>
<td>
<p>-5<br>
-2<br>
-7<br>
1<br>
91<br>
3<br>
8<br>
6<br>
4<br>
-4<br>
-1<br>
0<br>
2<br>
1111<br>
7<br>
5<br>
92<br>
93<br>
-6<br>
-3<br>
12</p>
</td>
</tr>
</table>
<h3>Annex 10: initial presentation layer</h3>
<table border="1" cellspacing="3" cellpadding="0" cols="2">
<tr>
<td>
<p>constructor RecordFormatter(RecordInfo)</p>
</td>
<td>
<p>initializes the formatter from a given metadata</p>
</td>
</tr>
<tr>
<td>
<p>void writeInPage(Writer stream, Dictionary data)</p>
</td>
<td>
<p>writes a view of the given data to the given stream</p>
</td>
</tr>
<tr>
<td>
<p>void writeInForm(Writer stream, Dictionary data)</p>
</td>
<td>
<p>writes an edit form of the given data to the given stream</p>
</td>
</tr>
<tr>
<td>
<p>void writeInFormNew(Writer stream)</p>
</td>
<td>
<p>writes an empty form (with default values) to the given
stream</p>
</td>
</tr>
<tr>
<td valign="top">
<p>Dictionary readFromQueryString(Dictionary query)</p>
</td>
<td valign="top">
<p>reads the field from the http query<br>
The query object can always be constructed as a Dic from any query
string (even with multiple values per field)</p>
</td>
</tr>
</table>
<p>adjustment methods (less urgent):</p>
<ul>
<li>formatting: adjustField(fieldname,
"&lt;b&gt;$description:&lt;/b&gt; $value") etc</li>
<li>field order change: moveBefore(field1, field2), etc</li>
<li>field hiding/keeping: hideField(fieldname)</li>
</ul>
<p>later:</p>
<table border="1" cellspacing="3" cellpadding="0" cols="2">
<tr>
<td valign="top">
<p>void writeInFormSearch(Writer stream)</p>
</td>
<td valign="top">
<p>writes a search form<br>
Still to be thought out (depending on how searches will be
represented internally)</p>
</td>
</tr>
<tr>
<td valign="top">
<p>void writeInFormFormat(Writer stream)</p>
</td>
<td valign="top">
<p>opens the formatter for editing to a non-tech user, as a edit
form</p>
</td>
</tr>
<tr>
<td>
<p>void writeInPageFormat(Writer stream)</p>
</td>
<td>
<p>pens the formatter for editing to a non-tech user, as a page</p>
</td>
</tr>
</table>
<ul>
<li>according to the RecordHandler conventions, the <b>writeIn*</b>
methods can be implemented as <b>RecordHandler.writeAll</b> (which
RecordFormater will inherit) if each field type is implemented as a
metadata.html.FieldFormatter (having more or less the same methods
as above). Thus, there will be intFormatter, intEnumFormatter,
charFormatter, charEnumFormatter, dateFormatter, etc</li>
<li>we should think of cool demos</li>
</ul>
<h4>issues and dilemmas</h4>
<ul>
<li>fast development: besides having html.RecordFormatter, one
should be able to get html very fast once metadata files URLs links
like</li>
</ul>
<p>/best/student/$edit<br>
/best/student/$view<br>
/best/student/$new<br>
/best/student/$list</p>
<p>are a tempting way to go. Then one just has to pass a pointer
(for $view and $edit) to get results, which should be the defaul
for $list views. A generic servlet interpreting this (reading the
path to the metadata from its url path, reading the pointer and the
form data from the query string, etc) can handle all cases needed,
generating automatically some jhtml/jsp source for each case, which
will then be adjusted by the webmaster.</p>
<p>Problems:</p>
<ul>
<li>one has to provide several such addresses for one metadata, one
for each actor or task. Probably something like</li>
</ul>
<p>/organiser$/best/student/$view<br>
then organiser could be a search in the auth and student tables
that would give a non-empty result if the respective is an
organiser</p>
<ul>
<li>how does a non-technical user adjust the formatter (change
field order, hide fields, etc)?</li>
<li>a easy-to-do way, but quite technical, is a <b>fo <a name=
"format"></a>rmat()</b> method in the servlet (jhtml, jsp, etc),
that would be called once, at servlet initialization. That method
would call various methods of the default formatter (for the
respective RecordInfo) to adjust it. Then the servlet service code
(or the jsp/jhtml page) will just call</li>
</ul>
<p>getFormatter().format(getData())</p>
<p>where getFormatter returns the formatter adjusted in the
format() method, and format() returns the data identified by the
pointer (for $edit and $view) or the bunch of data selected by
$list.</p>
<ul>
<li>a not that easy to-do, but very user friendly is a html form
that would open for URLs like</li>
</ul>
<p>/best/student/$view$format</p>
<ul>
<li>a more webmaster-like technique: we can generate jhtml/jsp
containing formatting calls and let the site owner format it. That
has some problems:</li>
<li>how/where to format new fields?</li>
<li>how to format fields uniformly?</li>
</ul>
<ul>
<li>search forms can of course be generated from the metadata info.
In the structure above, one would have /best/student/$search</li>
<li>passing pointers via CGI: sv=xx&amp;uid=yy can be too obvious,
a pointer registry can exist, where all Records are registered by
hashCode, then the hashCode is passed and used as a password to get
the Record back</li>
<li>in general, one can think about encoding the URL-passed data,
at a later stage</li>
<li>logging at http level?</li>
</ul>
<h3>Annex: <a name="JSP-Parallel_java"></a>JSP-Parallel java
code</h3>
<p>This proposal was made for a fast implementation process and to
give a well-known place for the java code.</p>
<p>Every JSP page that uses the Makumba taglibs will create a
parallel Java class per operation invoked. The first page execution
will call default implementations of the methods in the class and
may announce the developer where to go to edit the code. The
"pointer" to the code can stay in the generated HTML as a comment.
It is intended that when the parallel code changes, the JSP is
touched and re-compiled automatically. The JSP will call dedicated
callbacks in the parallel class at different moments in the
execution as illustrated below. References to the page and to the
"brother classes" generated by the page will be available.</p>
<p>For example, a typical edit form will invoke the operations
"writeInForm" and "edit" (since there are no response pages) So it
will generate two parallel classes. Still, in such a case
(presentation-response), one might have a single class dealing with
the whole transaction, since the two are logically linked.</p>
<p>Still need to consider<br>
- the webmaster-security of the solution<br>
- the mechanisms by which the objects from the parallel class are
created and referenced<br>
- our last dilemma in Coimbra: should the access rights and field
access rights be expressed in JSP-parallel or in DD-parallel (one
java class per Data description, i.e. Makumba file) java code?</p>
<ul>
<li>The argument for the DD-parallel is that access rights should
be changed in only one place. One counter-argument is that if the
access rights changes, the JSP pages that count on them have to be
changed anyway</li>
<li>The JSP-parallel pro is that they seem more practical. Only one
(rather than two) set of java classes is needed, which makes clear
where the application code is. On the other hand, it's not clear
what happens with doing the operation on a sub-table (set or
pointer). One way to do it is to generate a parallel-class for that
operation as well.</li>
<li>the DD-parallel can be replaced by a more powerful DD
grammar</li>
<li>instead of changing the access rights as such, one can change
the definition of the actor who has the respective rights.</li>
<li>We should probably play with both and see what's more practical
for BEST and Karamba purposes. We should define the Karamba
applications (the present state and what we suspect for the new
state) and see what we need.</li>
</ul>
<p>- the clear definition of the methods. Below are some guidlines,
but the final interface will come out per operation (view, view in
form, edit, delete, insert) after some practice.</p>
<table border="1" cellspacing="3" cellpadding="0" cols="3">
<tr>
<td>
<p>method signature</p>
</td>
<td>
<p>semantics</p>
</td>
<td>
<p>default implementation</p>
</td>
</tr>
<tr>
<td>
<p>Object getInitialState()</p>
</td>
<td>
<p>returns the initial state of an operation implementation. the
state info will be passed to all methods of the template method</p>
</td>
<td>
<p>returns null</p>
</td>
</tr>
<tr>
<td>
<p>boolean isAuthorised(Pointer auth, Dictionary args, Object
state)</p>
</td>
<td>
<p>gets a pointer in an auth table detected by some http way<br>
returns whether the authenticated principal can be authorized as an
actor that can do the respective operation on the data</p>
</td>
<td>
<p>if you are in the JSP root, return true<br>
otherwise try to authenticate an actor with the type indicated by
the first directory. how that dir indicates the type is left open
for now</p>
</td>
</tr>
<tr>
<td>
<p>Object getData(Pointer auth, Dictionary args, Object state)</p>
</td>
<td>
<p>gets the data to view, view in form (pre-edit, pre-insert),
insert, or edit<br>
This is the typical place to do an <b>OQL search</b>. Having it
here will also allow for ordering and other presentation-related
issues.</p>
</td>
<td>
<p>one http arg is a pointer, data is retrieved from the main table
using it</p>
</td>
</tr>
<tr>
<td>
<p>boolean beforeExcecute(Object state)</p>
</td>
<td>
<p>Executes code before the operation (insert, delete, edit) is
executed. Returns whether the operation should be executed or
not.</p>
</td>
<td>
<p>empty</p>
</td>
</tr>
<tr>
<td>
<p>void afterExecute(Object state)</p>
</td>
<td>
<p>similar to above</p>
</td>
<td>
<p>empty</p>
</td>
</tr>
<tr>
<td>
<p>boolean needTransaction(Object state)</p>
</td>
<td>
<p>tells whether the operation and the rest should be wrapped in a
transaction</p>
</td>
<td>
<p>returns false</p>
</td>
</tr>
<tr>
<td>
<p>boolean canView( String field)<br>
boolean canEdit(String field)</p>
</td>
<td>
<p>field access level. <b>not clear if it will be here or
not.</b></p>
</td>
<td>
<p>returns true</p>
</td>
</tr>
</table>
<h3>Annex: Former explanation of folder structure... Should go to
some tutorial</h3>
<pre>
general/general_table1 (ex all countries, all languages)
       /general_table2 ...
organization1/major_table1  (ex best/student)
             /major_table2 ...
             /application1/table1 ...  (ex best/minerva/company)
                          /table2 ...
             /application2/table1 ...
organization2/major_table1 ...
common_application/table1 ...  (ex eventcalendar/event)
</pre>
<h3>Annex: BEST Data Definitions hierarchy</h3>
<p>This is what the Stockholm meeting had in mind for BEST
hierarchy. Object orientation and decoration were assumed</p>
<pre>
Auth
Student
best.Student decorates Student

Group
best.Lbg extends Group
best.Committee extends Group
best.vivaldi.Student decorates Student
best.vivaldi.Season
best.vivaldi.Activity
best.vivaldi.Application
best.minerva.Student decorates Student

Company
best.minerva.Company decorates Company
best.minerva.CompanyRepresentative
best.karamba.Document
best.karamba.Message
best.karamba.Repository
best.karamba.OfficialArchive extends Repository
best.karamba.CommitteeArchive extends Repository
best.karamba.TigroBlabla extends CommitteeArchive
</pre>
<script type="text/javascript">
   makeFooter("$Id$");
</script>
</body>
</html>
